Title: A Holistic Framework for Knowledge Discovery and Management
Authors: DELEN, DURSUN., AL-HAWAMDEH, SULIMAN
Subject: Knowledge management
Publish: 2009
Status: full text
Source: Communications of the ACM; Jun2009, Vol. 52 Issue 6, p141-145
Preparation: Scientific Database Management Journal Articles www.SYSTEM.parsiblog.com
Abstract: The article discusses the discovery and management of information. The authors propose an integrated, holistic framework for knowledge management that is composed of interdependent modules meant to perform activities involved with the knowledge management cycle. Such activities, the authors state, include storing, creating, and extracting knowledge. Topics of discussion include exploiting technologies such as data warehousing and data mining and assisting computer users with the problem of information overload.    --Download Article

Introduction: THE INCREASED INTEREST IN KNOWLEDGE DISCOVERY, knowledge management, and knowledge transfer can be attributed to many factors including the advances in information and communication technologies; data explosion and information overload; the expected significant loss in the workforce as the baby boomers retire; and, the need for organizations to better utilize their intellectual capital to stay ahead of the competition. As with the massive amounts of information being added to corporate databases and the Internet everyday, effective and efficient knowledge discovery and its management has become an imminent problem. In spite of addressing a special part of the problem, which has been the case in a vast amount of the recently published research articles on knowledge management, in this paper we propose a holistic framework for knowledge management. This highly integrated framework is composed of a number of interdependent modules designed to perform the activities of the knowledge management cycle including creating, extracting, storing and using/reusing knowledge. By exploiting various existing technologies such as data warehousing, data mining and text mining, along with Web crawling and federated search engines, this integrated system provides the knowledge worker with the most relevant information to make the best possible decision in a timely manner. This article emphasizes the importance of knowledge management, makes the case for an integrated knowledge management framework, displays and discusses a holistic knowledge management framework, elaborates on the required capabilities and functionalities of such a framework, and concludes with the future directions and final recommendations.
The information age that we are living in is characterized by rapid growth in the amount of data collected and made available in electronic media. This swift pace of growth in digitized information makes it imperative for us to seek alternative methods to effectively and efficiently utilize these invaluable assets.
In order to assist the user with the problem of information overload and not exacerbate it, these new methods should go beyond what is already available to the knowledge worker today. The most commonly used method for managing unstructured information has been to employ keywords indexing. While keywords are good at identifying information, they often fall short on telling how relevant such information is to the queried situation and might lead to the assumption that the more often a keyword is mentioned the more relevant the information is to the query.
Most of the current information retrieval systems available on the Web today are based on keyword indexing with very little emphasis on context or textual information. There is a need for a more intelligent information retrieval system that can take into account the semantic information and go beyond the simple keywords searching. But given the limitation of artificial intelligence based systems and the fact that most knowledge exists in the minds of translapeople in the form of tacit knowledge, it is imperative to build collaborative information systems capable of integrating this valuable knowledge into the information retrieval environment.
To integrate tacit knowledge transfer processes into information systems there is a need to understand the complexity associated with tacit knowledge capture and transfer.5, 7, 10 Many believe that tacit knowledge residing within an individual is only observable via an individual’s actions.6, 7 But there have also been suggestions in tapping/harvesting tacit knowledge. Some of these methods include interview sessions, narrations or story telling, knowledge exchange protocols or action protocols using the repertory grid and analogies and metaphors.8 The discussion on whether tacit knowledge can be captured (codified) or transferred from one person to another will remain arguable as far as we continue to view knowledge in two dimensional space as either tacit or explicit. Al-Hawamdeh1 introduced a third dimension to the knowledge definition by distinguishing between actionable knowledge (tacit knowledge) in the form of behavior, skills, competencies and experiences and articulated knowledge (implicit knowledge) in the form of individual thoughts and language use. Viewing tacit knowledge as actionable knowledge and articulated knowledge help us in designing information systems that support the capture and transfer of these two types of knowledge. While articulated knowledge in the form of “Know How” and “Know Who” can be transferred easily using information systems technologies such as email, collaboration tools, discussion board and chat room technology, actionable knowledge on the other hand requires more advanced technologies such as video and other synchronous multimedia technologies. The challenge that most information and knowledge management systems face today is the lack of ability to integrate the capture and transfer of actionable knowledge, articulated knowledge as well explicit knowledge. In this paper, we report on the conceptual design of a highly interactive holistic framework for knowledge management. The main idea behind the system can be explained in a three-step process.
First is to try and satisfy the user request by lotors cating and extracting information from diverse information resources. At this end, technologies such as text mining, web crawlers and sophisticated search engines are used to identify and locate relevant information and make it available to the user.3, 4 In this step the system is dealing with explicit knowledge or information stored in databases, data warehousing and on the Web. If the user is still not satisfied with the outcome from the first step, then the next step is to engage an intelligent broker that can match the user request with pre-identified expertise and post the user request to an expert. At this stage we are dealing with a type of knowledge in the form of “Know How” that can be articulated and captured using tools such as an email system, discussion board or any other collaborative tools. If all that failed and the user request still is not addressed, then we might be dealing with an actionable knowledge that other means of communication is necessary. Actionable knowledge is a type of knowledge that requires face-to-face interaction, apprenticeship, and socialization. In this case, the intelligent broker could suggest ways of addressing the user request by facilitating that interaction to take place.

Integrating Diverse Information Sources
One of the biggest problems that organizations are facing today is the lack of ability to capture and integrate information residing in diverse sources, some of which are internal to the organization (for example, transaction databases, data warehouses, knowledge portals, and document management systems) while others are external (for example, commercial databases, credit reports, market research excerpts, news agency announcements, Web pages, etc.). Tightly integrating multiple diverse sources into a single cohesive system for the sake of centralization of the information sources often creates rather large extremely rigid systems that are not practically manageable.
The alternative method to the tightly integrated systems is the use of federated systems, which are based on a loose interfacing of applications and data resources.2 However, since most federated systems employ a central data management dictionary with  that convert search queries into something that can be understood by the diverse information sources, without frequent updates to the dictionary and the translators, the entire data sources can quickly become obsolete and unusable for searching. Additionally, as the number of source systems becomes larger, the maintenance of the dictionary and translators becomes harder, if not insurmountable.
The other issue with integrating information from multiple sources is the high cost associated with such a process. Information only becomes valuable when people need it and can make use of it. Therefore, whether centralized or federated the construction of any large information repository should be driven by the obvious needs of the users. That is, creating a knowledge repository to store information that is often used or sought after is normally a good idea, as opposed to creating knowledge repositories because we can technically and financially afford to do so. Thanks to the increasingly lower cost of storage devices and the advances in data warehousing technology, large amounts of information collected over time can now be retained for a longer period of time. Some of the information retained today might be for legal purposes, but the majority of it is retained with the ive of mining it in the hope of discovering valuable knowledge that can be used to assist in the decision making processes to enhance an organization’s competitiveness. According to recent trends, extracting useful information from large amounts of diverse information sources requires the use of data and text mining tools. Data Mining is the process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data stored in structured databases, where the data are organized in records structured by categorical, ordinal and continuous variables. However, a vast majority of business data are stored in documents that are virtually unstructured.
According to a recent study by Merrill Lynch and Gartner, up to 90% of all organizational data is stored in some sort of unstructured text.9 Therefore there is a need for tools to process unstructured data sources. Text mining (also known as text data mining and knowledge discovery in textual data bases) can be described as the process of deriving novel information from a collection of texts (also known as a corpus) which aims to address this need. By novel information, we mean associations, hypotheses or trends that are not explicitly present in the text sources being analyzed. Even though text mining is considered a part of the general field of data mining, it differs from regular data mining, such that in text mining, the patterns are extracted from natural language text rather than from structured databases of facts. Databases are designed for programs to process automatically; text is written for people to read and understand.
We do not have programs that can “read” and “understand” text (at least not in the manner that human beings do). Furthermore, despite the phenomenal advances achieved in the field of natural language processing, we will not have such “intelligent” programs for the foreseeable future. Many think that it will require a full simulation of how the mind works before we can write programs that read and understand the way people do. If so, then what does text mining do? On the most basic level, it numericizes and characterizes the unstructured text documents into structured table representations and then, using data mining tools and techniques, extracts patterns/knowledge from them.

A Holistic Framework for Knowledge Discovery Management
The limitations of the current keyword based information retrieval systems and the complexity associated with integrating information from diverse information sources intensify the search for a better inquiry system, which is capable of taking advantage of the latest technologies in knowledge discovery and knowledge re-use. Since technology is merely an enabler in knowledge management environment, there is a need to achieve a certain degree of harmonization between people, technology, and information. Figure 1 shows the main components of the integrated knowledge management platform. The platform is divided into two distinct subsystems (knowledge creation sub-system and knowledge utilization subsystem) that are integrated into each other through the knowledge depository. The level of integration between these subsystems (and the modules within them) determines the performance of the whole system and the quality of services provided to the users. In the knowledge creation subsystem, the main components are the text and data mining tools and the WebCrawler. The text and data mining tools module is used to extract useful information from diverse information systems such as large data warehousing systems and textual information systems that includes the Web. Extracted information in the form of knowledge nuggets is then stored in the knowledge depository to be used by the knowledge utilization subsystem.
The WebCrawler is an intelligent agent used to monitor relevant information on the Web and captures and stores information into the knowledge depository in the form of knowledge nuggets which are updated for accuracy at all the times. The knowledge depository is a key component of the knowledge management system. It stores and organizes information coming from the modules within the knowledge creation subsystem as well as capturing the transactional logs of questions and answers handled by ask the expert module. Organizing information in a way that makes it easily and quickly retrievable is the key for any successful information system, since the choices made for the system organization in the knowledge depository will have a major impact on the performance of the whole knowledge management system. The Intelligent Broker component of knowledge utilization subsystem is designed to facilitate collaboration and interaction between the user community and knowledge depository as well as the human experts (which are identified by the system via the “know-who” knowledge nuggets within the depository).
With the integrated components such ask the expert, knowledge portal, problem profiling, and personalization, the knowledge utilization subsystem provides users with access to the most relevant information via the knowledge depository and/or direct contact to human experts. It will also provide direct access to the knowledge depository via the ad-hoc keyword searches.

Knowledge Discovery and Re-Use
One of the ways to minimize knowledge loss is to enhance knowledge capture and re-use within the organization. Knowledge loss within the organization could happen as a result of many internal and external factors. Some of these factors include retirements, resignations, and restructuring or even outsourcing activities.11 The challenge in minimizing knowledge loss is the ability to identify the knowledge sources and then take the necessary measures to ensure knowledge retention and utilization. Unlike information, where information systems can be used to collect process and store information, most knowledge resides in the minds of people and therefore it is necessary to involve these people in the process.
This means that we acknowledge the fact that certain types of knowledge are not codifiable and the best way to tap into that type of knowledge is to create a knowledge management environment by which these sources of valuable knowledge are identified and made available as and when the knowledge is needed.
The inquiry platform is an integrated environment by which the users interact with different types of knowledge including the human sources in the form of domain experts. Since the cost associated with eliciting knowledge from the experts is much higher than finding it in structured databases or on the Web,12 the inquiry system has to be designed in away that users are directed to the human expert as a last resort, and in the event the information retrieved from the knowledge depository is not satisfactory and/or does not answer the user’s queries. Figure 2 shows the workflow for posing a query or an unstructured question to the system. When the user submits a new query, the inquiry system first tries to determine if the new query matches any of the previously processed queries and/or the questions answered by the subject experts in the knowledge depository....    --Download Article